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CONFIDENCE  INTERVALS  FOR  THE  RELIABILITY  OF  A  FUTURE  SYSTEM  CONFIGl'RAT!  ''N 
1.  INTRODUCTION 

Reliability  growth  management  is  a  critical  function  in  the  develop,  -f 
programs  of  major  defense  systemsJ  It  consists  of  planning,  monitoring,  and 
controlling  the  growth  of  reliability  parameters  throughout  system  development 
in  order  to  achieve  the  reliability  milestones  for  each  test  phase  and  for  the 
overall  program.  A  key  factor  in  this  process  is  the  ability  to  assess  the 
risk  of  not  meeting  a  reliability  requirement  and  to  make  such  assessment  at 
an  early  stage  in  the  current  test  phase.  If  this  risk  is  unacceptably  high, 
the  program  manager  may  then  have  an  opportunity  to  take  remedial  action 
before  test  time  or  other  program  resources  are  exhausted.  The  risks  of  fail¬ 
ing  to  achieve  program  goals  or  contractual  requirements  can  therefore  be 
minimized.  Instead  of  having  to  react  to  program  shortcomings  after  the  fact, 
management  can  exert  positive  control  over  the  growth  process  to  accomplish 
reliability  objectives. 

Reference  1  (pp.  10,  23,  28,  64-66,  75-78)  discusses  the  use  of  reliability 
growth  models  to  project  reliability  estimates  beyond  the  present  test  time  to 
some  future  time,  such  as  the  end  of  the  current  test  phase.  These  projections 
are  valid  only  if  test  conditions  remain  relatively  constant  and  the  develop¬ 
ment  effort  continues  at  its  previous  level.  The  projected  reliability 
estimates  are  compared  with  future  milestones  in  order  to  assess  whether  the 
reliability  enhancement  program  is  likely  to  reach  a  successful  conclusion. 

One  of  the  problems  with  assessing  a  program  by  this  method  is  how  to 
evaluate  the  accuracy  of  the  reliability  projections.  Such  projections  are 
only  point  estimates  and  do  not  reflect  the  uncertainties  that  accompany 
random  sampling  from  a  probabilistic  model.  In  this  paper  we  show  how  to 
quantify  these  uncertainties  when  the  Weibull  process  is  used  to  model  and 
forecast  reliability  growth.  The  result  is  an  objective  appraisal  of  current 
program  risks,  and  this  appraisal  can  be  factored  into  those  management 
decisions  which  may  impact  on  future  reliability  parameters. 

The  Weibull  process  model  has  been  successfully  applied  to  the  reliability 
test  results  of  many  complex  defense  systems.  It  is  introduced  ir,  Section  2 
in  a  parametric  form  that  is  especially  suited  to  the  problem  of  forecasting. 
The  basic  features  of  this  model  are  described  in  Appendix  C  of  Reference  1, 
which  includes  confidence  interval  procedures  for  the  reliability  of  the 
current  system  conf iguration.  (See  also  References  2  and  3.)  The  theory 
developed  in  Section  3  extends  these  latter  results  to  provide  inferential 


^Department  of  Defense,  Reliability  Growth  Management,  Military  Handbook  189, 
Naval  Publications  and  Forms  Center,  Philadelphia,  PA,  February  1981. 

2Bain,  L.  J.  and  M.  Engelhardt,  "Inferences  on  the  Parameters  and  Current 
System  Reliability  for  a  Time  Truncated  Weibull  Process,"  Technometrics , 

Vol .  22,  pp.  421-426,  August  1980. 

3Crow,  L.  H.,  Confidence  Interval  Procedures  for  Reliability  Growth  Analysis, 
Technical  Report  No.  197,  U  S  Army  Materiel  Systems  Analysis  Activity, 
Aberdeen  Proving  Ground,  MD,  June  1977. 
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procedures  for  future  reliability  levels.  These  procedures  are  illustrated 
in  Section  4,  where  confidence  intervals  are  obtained  for  the  reliabilii  to 
be  achieved  at  future  points  in  a  test  phase  which  is  still  in  progrtss.  c 

obtained  by  an  equivalent  technique  is  the  risk  of  not  achieving  a  certain 
reliability  level  at  the  end  of  the  test  phase. 

2.  SPECIFICATION  OF  MODEL 

Consider  a  reliability  growth  test  phase  which  has  been  underway  for  T 
units  of  testing.  We  shall  hereinafter  regard  these  test  units  as  time, 
although  they  could  equally  well  represent  other  units  such  as  distance. 
Suppose  that  the  test  phase  began  at  time  0,  but  is  planned  to  continue  for 
an  additional  S  units  of  testing  till  test  time  T+S,  at  which  point  the  system 
configuration  will  have  failure  rate  R.  Our  objective  is  to  make  inferences 
about  the  parameter  R. 

A  Weibull  process  is  a  nonhomogeneous  Poisson  process  with  an  intensity 
function  that  can  be  expressed  as  a  multiple  of  some  power  of  the  test  time. 
For  the  particular  test  phase  described  above,  an  intensity  function  of  the 
appropriate  parametric  form  is 

r(t)  =  R[t/(T+S)]6-1,  (1) 

where  R>0,  6>0,  and  0<t<T+S.  As  shown  in  Figure  1,  the  function  r(t)  models 
the  failure  rate  of  the  system  configuration  as  it  changes  over  a  reliability 
growth  test  phase  of  length  T+S,  and  the  failure  rate  at  the  end  of  the  (as 
yet  uncompleted)  test  phase  is  given  by  r(T+S)  =  R. 

The  failure  rate  model  in  Figure  1  shows  a  decreasing  trend  during  future 
testing  from  time  T  to  time  T+S.  This  trend  reflects  our  previously  stated 
intention  to  continue  reliability  improvements  throughout  this  period.  The 
case  in  which  reliability  is  constant  from  T  to  T+S  is  treated  in  Reference  4. 

According  to  the  scenario  of  this  paper,  test  results  are  available  for 
the  test  period  from  time  0  to  the  (current)  time  T,  but  the  system  testing 
from  time  T  to  time  T+S  has  not  yet  been  accomplished.  Let  N  be  the  number 
of  failures  that  occur  before  time  T  and  T.| ,  ...,  T^  the  observed  failure 

times  (0-T1  <  ...  <TN<T).  Then  the  Poisson  process  with  intensity  function 

r(t)  has  a  sample  function  density  given  by 

fpi  T  T  (ft  »  t.  ,  .  .  •  ,  t  ) 

N,Tj ,  ...»  Tjy  1  ft 


4Miller,  G. ,  "Efficient  Methods  for  Assessing  Reliability,"  Proceedings  of  the 
Nineteenth  Annual  U  S  Army  Operations  Research  Symposium,  Part  III,  pp.  33-42, 
October  1980. 
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exp[-(RT/6)Q1'6] 


if  N  =  0, 


'2.1) 


Rn  n  C(T+S)/t-]1-eexp[-(RT/B)Q1 if  N  =  n>0,  (.  .  ) 

i=l  1 


where  Q  =  (T+S)/T;  n  =  0,  1,  . . . ;  and  0<t]  <. . .  <tn<T.  (See  e.g.,  Reference  5.) 


RELIABILITY  GROWTH  TEST  PHASE - ► 


FIGURE  1.  Intensity  Function  for  the  Case  /9<1. 


^Snyder ,  D.  L. ,  Random  Point  Processes,  John  Wiley  and  Sons,  New  York.  NY. 
1975. 
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3.  DERIVATION  OF  RESULTS 
? . 1  Point  Estimators 

The  Weibull  process  model  is  used  in  applications  where  Pr(N  -  ' 
quite  small,  and  therefore  the  likelihood  expression  in  Equation  (2.2; 
maximized  to  obtain  point  estimators  for  8  and  R  as  follows: 

N 


N/  l  1 n(T/T . ) , 

(3) 

i=l  1 

N8Qe-1/T. 

(4) 

As  would  be  expected,  the  expression  in  (3)  is  identical  to  the  estimator  for 
8  in  Reference  2  (Equation  (4)).  The  projected  mean  time  between  failures 
(MTBF)  for  the  system  configuration  at  the  end  of  the  test  phase  (time  T+S)  is 

estimated  by  R  . 

The  point  estimators  in  Equations  (3)  and  (4)  are  convenient  because 
of  their  simplicity,  but  were  obtained  without  conditioning  formally  on  the 
event  NO.  As  a  practical  matter,  inferences  on  the  two-parameter  Weibull 
urocess  are  possible  only  when  N>0,  and  we  shall  condition  on  this  event  in 
the  sequel  without  further  mention. 

3.2  Reduction  of  the  Parameter  Space 


N 

Let  V  -  T  ln[ (T+S)/T . ] ,  and  observe  from  Equation  (2)  that  V  is  a 
i=l  1 

sufficient  statistic  for  8.  It  follows  from  Reference  6  (pp.  134-140)  that 
uniformly  most  powerful  unbiased  (UMPU)  hypothesis  tests  on  the  future  failure 
rale  R  can  be  constructed  by  utilizing  tue  conditional  distribution  of  N  given 
;-v.  To  obtain  this  distribution,  we  begin  by  determining  the  conditional 
distribution  of  V  given  N=n. 

Given  N=n,  the  random  variables  T. ,  ...,  T^  are  distributed  as  the 

order  statistics  from  n  independent  distributions  with  cumulative  distribution 

function 


t  T 

F(t)  =  /  r(x)dx/J  r(x)dx 
0  0 

=  (t/T)^, 


(5) 


;  Lehmann, "E.  L.  ,  Testing  Statistical  Hypotheses,  John  Wiley  and  Sons,  New  York, 

NY,  1959.  .  .  . . 
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where  0<t<T.  Let  X  be  a  random  variable  with  distribution  function  c.  r>  >_  i 
straightforward  calculation  shows  that  the  random  variable  ln[(T+S)/X] 
distributed  over  the  interval  (lnQ,<»)  according  to 

Pr{ ln[(T+S)/X]  <  y}  =  l-exp[-(y-lnQ)B],  ,r 

where  lnQ<y«*>.  This  latter  function  is  a  two-parameter  exponential  distri¬ 
bution  function  on  the  interval  (lnQ,°°).  The  conditional  distribution  of  V 
given  N=n  is  therefore  the  sum  of  n  such  distributions,  all  independent,  and 
consequently  is  a  three-parameter  gamma  distribution  with  density  function 


=  &n(v-nlnQ)n-1exp[-e(v-nlnQ)]/(n-l ) I ,  (7) 


where  nlnQ<v<°°. 

The  random  variable  N  is  Poisson  distributed  with  mean  value 
Sh(RT/b)Q^ so  that  (conditional  on  N>0) 

Pr(N=r.)  =  [l-exp(-e)]”^enexp(-9)/n! ,  (8) 

n  =  1,  2,  ...  .  Thus  the  joint  density  function  of  V  and  N  is 

fV,N(v,n)  =  fv!N(vln)  Pr(N=n> 


exp(-e-Bv) 

l-exp(-e) 


(RTQ) 


n  (y-nlnQ 
n! (n-1 


n-1 
_  # 


(9) 


where  n  =  1,  2,  ...  and  nlnQ<v«*>. 

In  the  case  S=0  (forecasting  zero  time  into  the  future),  we  see  that 
lnQ=0  and  that  the  results  in  this  paper  generalize  certain  results  in  [3] 
and  [2]  on  inferences  for  current  system  reliability.  In  the  case  S>0,  the 
above  inequality  nlnQ<v«»  implies  that  N  has  finite  support,  given  V=v: 

Pr(0<N<v/lnQ| V=v)  =  1.  (10) 

Given  V=v,  let  G(v,S)  be  the  greatest  integer  less  than  v/lnQ  if  S>0  and 

G(v,S )  =  «.  if  S  =  0. 

We  can  now  write  down  the  conditional  distribution  of  N  given  V=v  as 

p(n;R)  £  Pr(N=n|V=v,  N>0) 

.  (RTQ)n(v-n1nQ)n~1/n!(n-l):  (11) 

G(v ,S)  .  .  i 

l  (RT0)k  (v-klnQ)K_ ' /k I ( k-1 ) ! 
k=l 
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where  n  =  1,  2,  ....  G(v,S).  This  expression  for  p(n;R)  can  be  readily  1- 
uated  at  minimal  cost  with  an  electronic  computer. 

3.3  Inferential  Procedures 


A  conservative  1 -a  confidence  interval  for  R  can  be  constructed  by 

G( v  ,S)  n 

obtainina  values  R,  and  R„  which  satisfy  i  p(k;R, )  =  a1  and  £  p(k;R9)  -  u„ 

k=n  1  1  k=l  L  1 

where  =  a.  The  corresponding  confidence  bounds  for  R"^  (the  MTBF  at 

test  time  T+S)  are  R^"1  and  R-|’  .  Because  N  is  a  discrete  random  variable, 

construction  of  exact  confidence  intervals  would  require  rana^mization.  A 
UMPU  test  of  Hq:R<Rq  versus  H^:R>RQ  at  significance  level  a  calls  for  rejection 

G(v,S) 

of  Hn  if  l  p(k;Rn)  sa.  Other  UMPU  hypothesis  tests  can  be  constructed  in 
u  k=n  u 

a  similar  manner.  If  Rq"^  is  the  MTBF  goal  for  the  end  of  the  test  phase 
(time  T+S),  then  the  risk  of  not  achieving  this  goal  may  be  evaluated  as 

l  p(k;  Rq) • 
k=l  u 

4.  EXAMPLE 

Suppose  that  a  reliability  growth  test  phase  has  been  in  progress  for 
T=200  hours  and  is  scheduled  to  continue  for  another  S=200  hours.  From  the 
test  data  up  to  time  200,  we  wish  to  obtain  an  80  percent  confidence  interval 
for  the  MTBF  at  time  T+S  =  400.  The  following  failure  times  t^  were  recorded 

( n=21 )  :  2.2,  3.3,  4.5,  5.3,  5.8,  20.3,  27.4,  34.1  ,  55.2,  58.4,  61.4,  62.-, 

78.3,  78.4,  91.9,  97.7,  112.4,  116.9,  142.4,  176.8,  181.5. 

Equations  (3)  and  (4)  yield  6  =  .591  and  R-^  =  21.4,  and  it  is  also  of 
interest  to  observe  that  v/lnQ  =  72.3.  Thus  G(v,S)  =  72,  so  that  the  set  of 
positive  integers  less  than  or  equal  to  72  is  a  support  of  the  conditional 
distribution  of  N  given  V=v. 

With  Equation  (11)  we  obtain  by  iteration  the  values  R^  ^  =  12.7  and 

,  72  21 

R  '  =  38.6  such  that  l  p(k;R,)  =  .10  and  l  p(k;R0)  =  .10.  The  interval 

1  k=21  1  k=l  c 

(12.7,  33.6)  is  therefore  an  80  percent  confidence  interval  for  the  MTBF  at 
time  400. 

By  successively  taking  S=0  and  S=1 00 ,  we  can  obtain  in  a  similar  manner 
80  percent  confidence  intervals  (10.7,  26.0)  and  (11.9,  32.7)  for  the  MTBF  at 
times  200  and  300,  respectively.  All  three  confidence  intervals  are  shown  in 
Figure  2  for  comparison  purposes. 
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TEST  HOURS 


FIGURE  2.  Eighty  Percent  Confidence  Intervals  for  Current 
and  Future  MTBF  Parameters . 


Suppose  further  that  an  MTBF  goal  of  15.0  has  been  set  as  a  milestone  for 
the  end  of  the  current  reliability  growth  test  phase  (T+S  -  400).  Based  on 
the  data  up  to  time  T  =  200,  the  risk  of  not  achievinq  this  qoal  is 

;  p(k;15  )  =  .20.  In  view  of  such  a  result,  the  program  manaqer  should  feel 

k=l 

optimistic  about  this  aspect  of  the  development  program,  but  will  probably  want 
to  avoid  any  actions  which  might  adversely  affect  the  overall  reliability 
enhancement  effort. 

Next  page  is  blank. 

11 


REFERENCES 


Department  of  Defense,  Reliability  Growth  Management,  Military  Hanocv 
189,  Naval  Publications  and  Forms  Center,  Philadelphia,  PA,  February  . 

Bain,  L.  J.  and  M.  Engelhardt,  "Inferences  on  the  Parameters  and  Current 
System  Reliability  for  a  Time  Truncated  Weibull  Process,  Technometri , 
Vo'1,  22,  pp.  421-426,  August  1930. 

Crow,  L.  H.,  Confidence  Interval  Procedures  for  Reliability  Growth 
Analysis,  Technical  Report  No.  197,  U  S  Army  Materiel  Systems  Analysis 
Activity,  Aberdeen  Proving  Ground,  Maryland,  June  1977. 

Miller,  G. ,  “Efficient  Methods  for  Assessing  Reliability,"  Proceedings  of 
the  Nineteenth  Annual  U  S  Army  Operations  Research  Symposium,  Part  III, 
pp.  33-42,  October  1980. 

Snyder,  D.  L.,  Random  Point  Processes,  John  Wiley  and  Sons,  New  York, 

NY,  1975. 

Lehmann,  E.  L.,  Testing  Statistical  Hypotheses,  John  Wiley  and  Sons, 

New  York,  NY,  1959. 


Next  page  is  blank. 


13 


_ A 

DISTRIBUTION  LIST 

1 

No.  of 
Copies 

Organi zation 

■ 

1? 

Commander 

Defense  Technical  Information  Center 

ATTN:  DDC-TC 

Cameron  Station 

Alexandria,  VA  22314 

1 

Commander 

U  S  Army  Materiel  Development 
and  Readiness  Command 

ATTN:  DRCPA-S 

5001  Eisenhower  Avenue 

Alexandria,  VA  22333 

1 

r 

1 

Commander 

U  S  Army  Electronics  Research 
and  Development  Command 

ATTN:  DRDEL-AP-QA 

2800  Powder  Mill  Road 

Adel  phi,  MD  20783 

1 

Di rector 

U  S  Army  TRADOC  Systems  Analysis  Activity 

ATTN:  ATAA-SL 

White  Sands  Missile  Range,  NM  88002 

► 

1 

Director 

U  S  Army  TRADOC  Systems  Analysis  Activity 

ATTN:  ATAA-T 

White  Sands  Missile  Range,  NM  88002 

2 

Chief 

Defense  Logistics  Studies  Information  Exchange 

U  S  Army  Loqi sties  Management  Center 

ATTN:  DRXMC-D 

Fort  Lee,  VA  23801 

; 

l 

Commander 

U  S  Army  Concepts  Analysis  Agency 

8120  Woodmont  Avenue 

Bethesda,  MD  20014 

1 

1 

1 

Reliability  Analysis  Center 

ATTN:  Mr.  I.  L.  Krulac 

Griffiss  AFB ,  NY  13441 

15 

k  . 

- -  .  .  .  ^ 

- 1  J 

DISTRIBUTION  LIST  (Continued) 


Organization 
Aberdeen  Proving  Ground 
Director,  BRL,  Bldg  328 
Director,  BRL 

ATTN:  DRDAR-TSB-S  (STINFO  Branch) 
Bldg  305 

Director,  HEL,  Bldg  520 


